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[Name of the Document] Specification 
[Title of the Invention] PERFORMANCE INFORMATION 
REPRODUCING APPARATUS AND PROGRAM 
[Claims for the Patent] 
[Claim 1] 

A performance information reproducing apparatus 
characterized by comprising: 

file storage means that stores a musical tone data 
file, in which musical tone information is recorded, and 
a media data file, in which another type of media 
information is recorded, together with a management file 
in which reading manners of these data files are 
recorded; and 

reproduction data generating means that generates, 
based on the musical tone data file and the management 
file, reproduction data that designates the musical tone 
information and the media data file to be reproduced, in 
a same format as the reading of the musical tone 
information . 

[Claim 2] 

A performance information reproducing apparatus as 
claimed in claim 1, characterized by further comprising: 

reproduction data storage means that stores the 
generated reproduction data; 

reading means that reads the reproduction data from 
said reproduction storage means in accordance with 
reproduction clock for the musical tone by a musical tone 
information reproducing program; and 

reproducing means that reads the media data file 
designated by the read reproduction data from the file 
storage means, and independently reproduces the musical 
tone information in the reproduction data and the read 
media information in the media data file. 

[Claim 3] 

A performance information reproducing program for 
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causing a information processing apparatus which 
comprises file storage means that stores a musical tone 
data file, in which musical tone information is recorded, 
and a media data file, in which another type of media 
5 information is recorded, together with a management file 
in which reading manners of these data files are recorded, 
to execute a procedure, the procedure comprising: 

a step of generating, based on the musical tone data 
file and the management file, reproduction data that 
10 designates the musical tone information and the media 
data file to be reproduced, in a same format as the 
reading of the musical tone information; 

a step of storing the generated reproduction data; 

and 

15 a step of reading the stored reproduction data. 

[Detailed Description of the Invention] 
[0001] 

[Field of the Invention] 

This invention relates to a performance information 

20 reproducing system that reproduces a plurality of types 
of performance information (media information) in 
synchronism with each other, and in particular to a 
performance information reproducing system that are 
designed to generate reproduction data used to read out 

25 performance information to be reproduced in synchronism 
with each other. 
[0002] 

[Conventional Art ] 

Conventionally, there is known an apparatus which 

30 generates data for synchronized reproduction of a 

plurality of types of data, such as music and images, in 
timing in which musical performance is given and carries 
out synchronized reproduction of such various types of 
data based on the generated data, as disclosed by Patent 

35 Document 1, for example. 
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[0003] 

[Patent Document 1] Japanese Laid-Open Patent Publication 

(Kokai) No. H06-124093. 

[0004] 

5 However, the conventional apparatus generates such 

various types of data in advance so that the data will be 
reproduced in accordance with the timing of musical 
performance, and therefore there is the problem that it 
is difficult to change some of such data or to add a new 
10 type of data. 
[0005] 

[Problems to be Solved by the Invention] 

In view of the above mentioned problem, it is a main 
object of the present invention to provide a performance 

15 information reproducing system that are capable of 
realizing desired synchronized reproduction of a 
plurality of types of performance information (media 
information) , such as music and images with ease by 
easily generating reproduction data used to retrieve each 

20 of various performance information files in accordance 
with management data that specifies reproducing manners 
of the various performance information files and 
retrieving a file using the reproduction data when the 
synchronized reproduction is performed. 

25 [0006] 

[Means for Solving the Problems] 

According to the main features of the present 
invention, there is provided a performance information 
reproducing apparatus (PC, ED) comprising file storage 

30 means (3, 4F) that stores a musical tone data file (MF) , 
in which musical tone information is recorded, and a 
media data file (AF, VF) , in which another type of media 
information is recorded, together with a management file 
(MN) in which reading manners of these data files (MF-VF) 

35 are recorded, and reproduction data generating means (MPa, 
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VPa; M4-M10) that generates, based on the musical tone 
data file (MF) and the management file (MN) , reproduction 
data (RD) that designates the musical tone information 
and the media data file (MF-VF) to be reproduced, in a 
5 same format as the reading of the musical tone 

information [claim 1] , and a performance information 
reproducing program for causing a information processing 
apparatus (PC, ED)which comprises file storage means 
(3,4F) that stores a musical tone data file (MF) , in 

10 which musical tone information is recorded, and a media 
data file (AF, VF) , in which another type of media 
information is recorded, together with a management file 
(MN) in which reading manners of these data files (MF-VF) 
are recorded, to execute a procedure comprising a step 

15 (M4-M10; MPa, VPa) of generating, based on the musical 
tone data file (MF) and the management file (MN) , 
reproduction data (RD) that designates the musical tone 
information and the media data file (MF-VF) to be 
reproduced, in a same format as the reading of the 

20 musical tone information, a step (M4-M12; MPb, VPa, 2M, 
2V) of storing the generated reproduction data (RD) , and 
a step (MPd, MPe) of reading the stored reproduction data 
(RD) [claim 3] . Note that reference symbols or the like 
of corresponding embodiments are shown in the parentheses. 

25 [0007] 

Moreover, the performance information reproducing 
apparatus (PC, ED) may further comprise reproduction data 
storage means (M4-M12; MPb, VPa, 2M, 2V) that stores the 
generated reproduction data (RD) , reading means (MPd, 

30 MPe) that reads the reproduction data (RD) from said 

reproduction data storage means (2M, 2V) in accordance 
with reproduction clock for the musical tone (MC) by a 
musical tone information reproducing program, and 
reproducing means (MPd, APa, VPc) that reads the media 

35 data file (AF, VF) designated by the read reproduction 



data (RD) from the file storage means (3, 4F) , and 
independently reproduces the musical tone information in 
the reproduction data (RD) and the read media information 
in the media data file (AF,VF) [claim 2] . 
5 [0008] 

[Operation] 

In a performance information reproduction apparatus 
(PC, ED) according to the present invention, data files 
(MF, AF, VF) , in which various types of media information 

10 are recorded respectively, are stored in file storage 
means (3, 4F) together with management files (MN) for 
managing the reproduction manners of the data files (MF- 
VF) . Such media information includes musical tone 
information that is in a predetermined format, such as 

15 MIDI (Musical Instrument Digital Interface) standard and 
can be reproduced according to a musical tone information 
reproducing program, such as MIDI reproducing program, 
and one or more of other type media information that is 
reproduced according to corresponding reproducing program, 

20 for example sound wave information, such as audio data, 
or video information, such as video data. 
[0009] 

When reproduction of such various media information, 
as the performance information, is carried out, a 

25 reproduction data generating function (MPa; M4-M10) of 
the performance information reproducing apparatus (PC, 
ED) generates, in accordance with the management file 
(MN) , reproduction data (RD) which allows the musical 
tone information of the musical tone data file (MIDI file 

30 MF) and the other media data files (audio and video files 
AF, VF) to be read according to a musical tone 
information reproducing program (MIDI reproducing 
program) . Then, the musical tone information of the 
musical tone data file (MF) and each of media data files 

35 (AF, VF) are read out based on the reproduction data (RD) , 
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and the musical tone information and the other media 
information are independently reproduced by the 
respective media reproduction programs (MIDI reproduction 
program, audio reproduction program, and video 
5 reproduction program) (MPd, APa, Vpc) . 
[0010] 

Therefore, without carrying out difficult processing 
such as specially generating various media information in 
advance so that the media information is reproduced in 

10 accordance with reproduction timing of musical tone 
information, when reproduction is carried out, the 
management file (MINI) is used to easily generate 
reproduction data (RD) for reading the various media 
information and the various media information can then be 

15 read out in accordance with the reproduction data (RD) 

only by setting read manners (filenames, timing, and the 
like) for various media information in advance in a 
management file (MN) . As a result, synchronized 
reproduction of media information can be realized with 

20 ease. Further, an editing operation for a management 
file (MN) that specifies reproducing manners of the 
various media information is simple, so that by editing 
the management file (MN) , for example, to change some of 
the data files or add new data files, or to change the 

25 reproduction timing of respective media files, it is 

possible to flexibly set a desired reproducing manner. 
[0011] 

[Embodiments of the Invention] 

Embodiments of the present invention will now be 

30 described in detail, below with reference to the 

accompanying drawings. It should be noted that the 
embodiments described below are merely examples, and that 
a variety of modifications- are possible without departing 
from the scope of the present invention. 

35 [0012] 



[System Configuration] 

FIGS. 1 show examples of the entire construction of 
a performance information reproduction systems according 
to an embodiment of the present invention. In such a 
5 performance information reproduction system, it is 
possible to use any information processing apparatus 
insofar as it has a communication function and a 
performance information processing function. FIGS. 1 
show two typical examples of how such apparatus is 
10 applied. 
[0013] 

In an example of FIG. 1(1), a personal computer PC 
with a communication function is used in a performance 
information reproducing apparatus. Various media 

15 information (hereinafter also referred to as "performance 
information") such as musical tone information and image 
information is distributed from a server computer SV to 
the personal computer PC via a universal communication 
network such as the Internet. The personal computer PC 

20 is provided with control programs including a management 
program and a variety of dedicated media information 
reproducing programs corresponding to various kinds of 
media information. In accordance with these programs, 
the personal computer generates reproduction data (RD) , 

25 which enables synchronized reproduction of the various 
media information, and reads out the various media 
information based on the generated reproduction data (RD) 
and carries out synchronized reproduction of the same. 
[0014] 

30 Next, out of such reproduction outputs, image output 

is supplied to a display section of the personal computer 
PC so that images corresponding to the media information 
are generated, and sound output is supplied to a 
dedicated information processing apparatus EM dedicated 

35 to musical tones with a musical tone generating function, 
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such as an electronic musical instrument so that sounds 
corresponding to the media information are generated. It 
should be noted that the personal computer PC is equipped 
with a tone generator and a sound system or the like, and 
5 hence has a function of outputting sounds based on 
musical tone information or the like, so that the 
personal computer PC itself may generate sounds and 
images corresponding to the various media information 
without using a separate information processing apparatus 
10 EM for outputting sounds, such as an electronic musical 
instrument . 
[0015] 

On the other hand, in an example of FIG. 1(2), an 
information processing apparatus ED dedicated to musical 

15 tones, such as an electronic musical instrument, is used 
in a performance information reproducing apparatus. This 
information processing apparatus ED has not only a 
musical tone information processing function but also a 
general communication function and various media 

20 information processing functions that are based on 

control programs such as a management program and various 
kinds of reproducing programs. Various kinds of media 
information are distributed from the server SV via a 
universal communication network. In accordance with 

25 these programs, the information processing apparatus ED 
generates reproduction data (RD) which enables 
synchronized reproduction of the various media 
information, and reads out the various media information 
based on the generated reproduction data (RD) and carries 

30 out synchronized reproduction of the same to generate 
sound and images corresponding to the various media 
information . 
[0016] 

It should be noted that the performance information 
35 reproducing apparatus is not limited to the personal 
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computer PC and the information processing apparatus ED, 
such as an electronic musical instrument as shown in FIGS. 
1(1) and 1(2), but various types of information 
processing apparatuses that have the same functions as 
5 theirs can be used. In the embodiments described below, 
the case where the performance information reproducing 
apparatus (PC, ED) itself has a function of outputting 
sound is described. 
[0017] 

10 FIG. 2 is a block diagram showing an exemplary 

hardware construction of the performance information 
reproducing apparatus according to the embodiment of the 
present invention. This performance information 
reproducing apparatus is comprised of a central 

15 processing unit (CPU) 1, a random access memory (RAM) 2, 
a read only memory (ROM) 3, an external storage device 4, 
a detection circuit 5, a display circuit 6, a tone 
generator circuit 7, an effect circuit 8, a communication 
interface (communication I/F) 9, and the like, with these 

20 components 1 to 9 being interconnected via a bus 10. 
[0018] 

The CPU 1 that controls the entire apparatus 
executes various performance information reproduction 
processes in accordance with predetermined control 

25 programs. The RAM 2 functions as a processing buffer for 
temporarily storing various information used in such 
processes, and is comprised, for example, of a contents 
buffer (2C) , a reproduction data buffer (2M) , a video 
reproduction data (video control data) buffer (2V) , and 

30 the like. Control programs for causing the CPU 1 to 
execute the performance information reproduction 
processes and various data, tables, and the like are 
stored in the ROM 3. Such control programs include 
performance information reproducing programs such as a 

35 management program (CP) , a MIDI reproducing program (MP) , 
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an audio reproducing program (AP) , a video reproducing 
program (VP), and a score reproducing program (SP) . 
[0019] 

The external storage device 4 is a storage means 
5 that uses a storage medium such as a hard disk (HD) , a 
compact disc read only memory (CD-ROM) , a flexible disk 
(FD), a magneto-optical disk (MO) , a digital versatile 
disc (DVD) , or a memory card. Various control programs 
related to performance information reproduction and 

10 various data can be stored not only in the ROM 3 but 
additionally or alternatively in the external storage 
device 4. For example, when a control program is not 
stored in the ROM 3, by storing the control program in 
the external storage device 4, such as an HD or a CD-ROM, 

15 and loading the program into the RAM 2, the CPU 1 can be 
made to execute the same operation as in the case where 
the control program is stored in the ROM 3. It is also 
possible to implement the performance information 
reproducing apparatus by installing a performance 

20 information reproducing program as a control program in 

the external storage device 4, which is an HD or the like. 
By doing so, it becomes easy to add and/or to update or 
upgrade the control programs. 
[0020] 

25 The detection circuit 5 introduces operation 

contents of various operators of an operator device 11 
connected to the detection circuit 5 into the system. 
The operator device 11 is comprised of various setting 
panel operators such as a character keyboard and a 

30 pointing device (such as a mouse), and when the 

performance information reproducing apparatus is the 
electronic musical instrument ED, also includes 
performance operators enabling the user to input 
performance sounds . 

35 [0021] 
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The display circuit 6 controls display contents of a 
display section 12 connected thereto according to 
commands from the CPU 1. The display section 12 is 
comprised of a display for displaying various screens, 
5 such as a performance information reproduction screen. 

This performance information reproduction screen is also 
called "page screen", and the display state thereof is 
controlled in accordance with the management program (CP) , 
the video reproducing program (VP) , the score reproducing 
10 program (SP) , and the like. 
[0022] 

The tone generator circuit 7 outputs musical tone 
wave information corresponding to musical tone 
information, and the effect (applying) circuit 8, which 

15 is comprised of a DSP or the like, converts the musical 
tone wave information and sound wave information into 
desired sound signals. A sound system 13, which is 
comprised of a D/A converter, an amplifier, and a speaker, 
is connected to the effect circuit 8, so that sound 

20 signals based on the musical tone information and the 

sound wave information generated in accordance with the 
MIDI reproducing program (MP) and/or the audio 
reproducing program (AP) are converted into analog 
signals, which are amplified, and outputted as sounds by 

25 the sound system 13 . 
[0023] 

The communication I/F 9 is the interface (I/F) that 
connected to a universal communication network such as a 
local area network (LAN) , the Internet, or a telephone 

30 line, or connected to a MIDI network for communication 

with other MIDI apparatuses. Via these networks, various 
information can be sent to and received from external 
apparatuses as necessary. For example, various control 
programs and data can be obtained from an external 

35 information processing apparatus and musical tone 
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information such as MIDI data can be obtained from an 
external MIDI apparatus. For example, as described above, 
in the case where necessary control programs, data, or 
the like are not stored in the ROM 3 or the external 
5 storage device 4, such control programs, data, or the 

like can be downloaded from the server computer SV or the 
like via the communication network. 

[0024] 

[Contents] 

10 In the performance information reproducing apparatus 

according to the embodiment of the present invention, a 
number of contents files (hereinafter also referred to 
simply as "contents") for performance information 
reproduction are provided in a contents filing section 4F 

15 of the external storage device 4. These contents may be 
obtained from the server SV mentioned above or external 
information processing terminals, or may be generated by 
editing various data obtained by the present performance 
information reproducing apparatus. Also, in the case 

20 where contents files are stored in advance in the ROM 3, 
such contents files may be used for performance 
information reproduction . 
[0025] 

FIG. 3 shows the exemplary structures of contents 
25 files used for performance information reproduction and 
the management data in the contents files according to 
the embodiment of the present invention. In the 
following description, it is assumed that information 
according to MIDI standard is used as the musical tone 
30 information and the like. 
[0026] 

As shown in the leftmost column in FIG. 3, a 
plurality of contents files CF1, CF2, CFn are provided 

as contents for use in the performance information 
35 reproducing apparatus. As shown in the second column 
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from the left, the contents files CF each include a 
management file MN, and a plurality of types of media 
data files MF, AF, VF, and SF. It should be noted that 
the following description, the symbol "CF" represents the 
5 contents files as a whole, and the symbol "CFi" generally 
represents any single contents file. In the same way, 
the symbol "MNi" generally represents the management file 
of the contents file CFi. 
[0027] 

10 In the illustrated example, the media data files 

(hereinafter simply "media files' 7 ) are each comprised of 
a MIDI file (musical tone information file) MF, an audio 
file (sound wave information file) AF, a video file 
(image information file (VF) , and a score file SF. A 

15 single contents file CF includes a plurality of MIDI 
files MF11, MF12, audio files AF11, AF12, video 

files VF11, VF12, ... and score files SF11, SF12, ... (each 
reference symbol corresponds to the first contents file 
CFI) . It should be noted that the following description, 

20 the symbols "MF", "AF", "VF", and "SF" represent 

respective media files as a whole and the first suffix 
"i" that follows these symbols "MF" , "AF" , "VF", and "SF" 
generally indicates the media file corresponding to any 
contents file CFi. 

25 [0028] 

As shown in the third column from the left in FIG. 3, 
a management file MN for each piece of contents (there is 
one management file per each piece of contents) is 
comprised of a plurality of page data PD1, PD2, ... with 

30 each page data having a plurality of group data Gdll, 
Gdl2, Gd21, Gd22, .... Each group data (one 

reproduction part data) Gd is set on a group-by-group 
basis, each group (k) being a unit by which a user can 
separately designate media files to be reproduced using 

35 that group data Gd. Each group (k) is also referred to 
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as "one reproduction part". 
[0029] 

Each group data (single reproduction part) Gd 
represents a "blueprint" of reproduction data in one 
5 reproduction part expressed in a user-definable page 
description language, such as XML (extensible Markup 
Language) and, as shown in an upper part of the rightmost 
column in FIG. 3, is comprised of control instruction 
(display instruction) information Cd and a plurality of 

10 media file designation information Md, Ad, Vd, and Sd. 

The media file designating information Md, Ad, Vd, and Sd 
are for designating media files MF, AF, VF, and SF to be 
reproduced corresponding to that group data (one 
reproduction part) Gd on a group-by-group (reproduction 

15 part) basis. 
[0030] 

The control instruction information (display 
instruction information) Cd includes display instruction 
information designating a display state such as display 

20 size and display position for each of data designated for 
reproduction by the present group (present reproduction 
part) (k) to be displayed on a page screen of the display 
section 12. The control instruction information also 
includes display data for the title of the group, brief 

25 comments, and the like. The display data may include the 
name and initial still image of a media file to be 
displayed first in a display area (window) for the 
designated media data. The control instruction 
information Cd may also include setting information such 

30 as timing setting information for setting MIDI clock 

timing in the case where information, such as MIDI file 
designation information Md that specifies MIDI clock 
timing is not present . 
[0031] 

35 In the illustrated example, the MIDI file 
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designation information Md, audio file designation 
information Ad, video file designation information Vd and 
score file designation information Sd are included as the 
media file designation information. A MIDI file, an 
5 audio file, a video file, and a score file MF, AF, VF, 

and SF to be reproduced are designated by the media file 
designation information Md, Ad, Vd, and Sd on a group 
(reproduction part) (k) on a group-by-group (reproduction 
part) basis. 
10 [0032] 

As shown in the lower part of the rightmost column 
in FIG. 3, out of the media file designation information, 
the audio file designation information and video file 
designation information (for one reproduction part) Ad, 

15 Vd that designate audio and video files AF, VF are 

comprised of pairs of designated filename data Fnl, Fn2, 

and timing data Tml, Tm2 indicative of timing in which 
reproduction of the audio file and the video file AF, VF 
designated by the filename data Fnl, Fn2, ... is to start, 

20 with such pairs being arranged in the order of 

reproduction sfart timing. Each timing data Tml, Tm2, ... 
can designate the reproduction start timing of the 
designated video file VF using any desired method, such 
as by designating a number of bars and a number of beats. 

25 [0033] 

Next, FIG. 4 shows the exemplary structure of each 
media file (single file) . Each contents file CFi 
includes a plurality of types of media files. As shown 
in the left column in FIG. 4, for example, the types of 

30 media files are the MIDI file, the audio file, the video 
file, and the score file MF, AF, VF, and SF as mentioned 
above. These media file types are each comprised of a 
file group MFi, AFi, VFi, or SFi of a plurality of files 
MFil, MFi2, ... , AFil, AFi2, ... , VFil, VFi2, ... , or SFil, 

35 SFi2, .... 
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[0034] 

As shown in an upper-left part of FIG . 4, each MIDI 
file (single file) MFia (where reference symbol "a" 
represents an arbitrary file in a file group i, and it is 
5 shown as the contents of a first MIDI file MFil(a=l) in 
the file group i in the drawing) is comprised of a master 
track CT and a reproduction track RT . The master track 
CT has recorded therein basic data (for example, tempo, 
key, and composer name) relating to musical tone 

10 information (musical composition information) of the MIDI 
file (single file) MFi . The reproduction track RT has 
recorded therein the contents of that musical tone 
information ^ (musical composition information). That is, 
as shown in the uppermost part of the right column in FIG. 

15 4 (in which the contents of the reproduction track RT of 
the first MIDI file MFil are shown) , the musical tone 
information (musical composition information) is 
comprised of pairs of MIDI clock timing MG11, MC12, 
and MIDI events EV11, EV12, ... occurring in the MIDI clock 

20 timing MC11, MC12, ... are arranged in the order of the 
MIDI clock timing. 
[0035] 

As shown in the second part of the right column (in 
which the contents of the first audio file AFil in FIG. 4 

25 are shown), each audio file (single file) AFia is 

comprised of a plurality of audio samples ASH, AS12, 
AS13 that are arranged in the order of reproduction, with 
the consecutive audio samples ASH, AS12, AS13 
representing sound information such as voices, instrument 

30 sounds, and natural sounds as waves. 
[0036] 

As shown in the third part of the right column (in 
which the contents of the first video file VFil in FIG. 4 
are shown) , each video file (single file) VFia is 
35 comprised of a plurality of video frames FR11, FR12, FR13, 
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... that are arranged in the order of reproduction. Each 
video frame FR11, FR12, FR13, ... may represent an image 

(scene) that is independent of other video frames, like a 
still image, or may represent an image (scene) that is 
5 related to preceding and succeeding frames as in a moving 
image (animation) . 

[0037] 

As shown in the lowest part of the right column (in 
which the contents of the first score file SFil in FIG. 4 

10 are shown), each score file (single file) SFia is 

comprised of pairs of display positions DP11, DP12, ... (of 
a changeable score display part) on a score represented 
by score information in the score file SF, and score 
events SE11, SE12, ... that occur at the respective 

15 corresponding display positions DP11, DP12, with the 

pairs being arranged in the order of display positions. 
[0038] 

[Setting Operation and Generation of Reproduction Data] 
Whenever performance information reproduction is 

20 carried out, the performance information reproducing 
apparatus according to the embodiment of the present 
invention enters a setting operation mode, in which 
reproduction data RD for synchronized reproduction of the 
media information in respective files MF, AF, VF, and SF 

25 is generated from the MIDI file MF and other types of 

media files such as audio, video, and score files AF, VF, 
SF. Then, the apparatus enters a reproduction operation 
mode, in which the reproduction data RD is used to 
reproduce the other media information in the other media 

30 files Af, VF, and SF in a reproducing manner according to 
the contents of the management file MN, in synchronism 
with reproduction of musical tones according to the 
musical tone information in the MIDI file MF. FIG. 5 is 
a functional block diagram [1] schematically showing 

35 functions used in the setting operation mode of the 
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performance information reproducing apparatus according 
to the embodiment of the present invention. In FIG. 5, 
broken lines represent parts that are activated to 
function in the reproduction operation mode (described 
5 later with reference to FIG. 12) . 
[0039] 

Now, a brief description will be given of the 
functions of the performance information reproduction 
system according to the embodiment of the present 

10 invention with reference to FIG. 5. In this performance 
information reproduction system, various types of media 
information (performance information) data files MF, AF, 
VF, and SF are stored in the contents filing section 4F 
or the like together with management files MN that manage 

15 the reproduction manners of the data files MF-SF. When 
synchronized reproduction of such media information is 
carried out, whenever reproduction is to be performed, 
first a reproduction data generating part MPa, etc. of a 
musical tone information (MIDI) reproducing program 

20 executing section MP generates, in accordance with the 

management file MN, reproduction data RD in the state for 
enabling musical tone (MIDI) data files and score data 
files MF, SF, and various other media (audio/video) data 
files AF, VF to be read out according to a musical tone 

25 information reproducing program. Then, reproducing parts 
of respective media information reproducing program 
executing sections MP, SP; AP, VP reproduce the musical 
tones and score information of the generated reproduction 
data RD and also the other media information of the other 

30 media data files AF, VF read out based on the 
reproduction data RD. 
[0040] 

This will be described more in detail below. As 
shown in FIG. 5, this performance reproduction system is 
35 comprised of a management program (browser program) 
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executing section CP that manages the progress of the 
respective media information, and a plurality of media 
information reproducing program executing sections that 
carry out reproduction operations for respective media 
5 information independently in accordance with instructions 
from the management program executing section CP. For 
example, these media information reproducing program 
executing sections are comprised of the above-mentioned 
MIDI reproducing program executing section, an audio 
10 reproducing program executing section, a video 

reproducing program executing section, and a score 
reproducing program executing section MP, AP, VP, and SP. 
[0041] 

The management program (browser program) executing 
15 section CP carries out management control of respective 

files in accordance with the management program stored in 
the ROM 3 to manage the contents buffer 2C provided in 
the processing buffer (RAM) 2. Also, during a setting 
operation by the performance information reproducing 
20 apparatus, a read processing section CPa, a distribution 
processing section CPb and a page screen display 
processing section CPc actively function. 
[0042] 

The MIDI reproducing program executing section MP 
25 operates in response to control instructions from the 
management program executing section CP to process 
performance information of a predetermined MIDI file MF 
in accordance with the MIDI reproducing program, and 
manages the reproduction data buffer 2M reserved in the 
30 processing buffer 2. The MIDI reproducing program 

executing section MP also performs control required for 
processing related to other media files AF, VF, and SF, 
and during a setting operation by the performance 
information reproducing apparatus, activates modular 
35 functions of the reproduction data generating section MPa 
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and a reproduction data storage section MPb. 
[0043] 

Based on control instructions from the management 
program executing section CP and information from the 
5 MIDI reproducing program executing section MP, the audio 
reproducing program executing section and the video 
reproducing program executing section AP, VP process, 
respectively, sound wave information of a designated 
audio file AF and image information of a designated video 

10 file VF according to the audio reproducing program and 
the video reproducing program. During a setting 
operation by the performance information reproducing 
apparatus, the audio reproducing program executing 
section AP does not have an activated module, but the 

15 video reproducing program executing section VP has a 

video reproduction data conversion/storage section VPa 
activated. The video reproducing program executing 
section VP also manages a video control data buffer 2V in 
the processing buffer 2. 

20 [0044] 

Based on control instructions from the management 
program executing section CP and information from the 
MIDI reproducing program executing section MP, the score 
reproducing program executing section SP processes score 

25 information , of a designated score file AF according to 
the score reproducing program. During a setting 
operation by the performance information reproducing 
apparatus, the score reproducing program executing 
section SP activates functions of a score display 

30 processing section SPa and a score reproduction data 
generating section SPb . 
[0045] 

Next, a description will be given of (1) a setting 
process carried out by the management program executing 
35 section CP, (2) processing by the respective reproducing 
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program executing sections MP, AP, VP, and SP in the 
setting operation mode, and (3) a MIDI generation process 
carried out by the MIDI reproducing program executing 
section MP in that order with reference to FIGS. 6 to 11. 
5 [0046] 

(1) Setting Process by Management Program Executing 
Section CP 

The setting process by the management program 
executing section CP is carried out during a setting 

10 operation by the performance information reproducing 

apparatus. A user sets the apparatus into the setting 
operation mode and designates a desired contents number 
(i) and a page (j) in the contents (i) by operating a 
predetermined operator of the operator device 11 in 

15 response to a contents menu with a tree structure that 
has been displayed in a guide screen on the display 
section 12. Responsive to this designation, the read 
processing section CPa of the management program 
executing section CP first reads a single contents file 

20 CFi corresponding to the designated content number "i", 
out of the contents files CFI to CFn recorded in the 
contents filing section 4F of the external storage device 
4 and a contents storage area of the ROM 3, that is, 
reads a management file MNi, MIDI files MFi, audio files 

25 AFi, video files VFi, and score files SFi included in the 
contents CFi and stores the files in the contents buffer 
2C. 

[0047] 

Next, the distribution processing section CPb 
30 initializes all group data Gdjl, Gdj2, ... included in page 
data PDj corresponding to the designated page j to 
display the initialized data in the page screen display 
processing section CPc. That is, the data distribution 
processing section CPb reads out all the group data Gdjl, 
35 Gdj2, ... of the page j from the management file MNi of the 
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contents file CFi stored in the contents buffer 2C and 
sends the readout data to the page screen display 
processing section CPc. 
[0048] 

5 The page screen display processing section CPc 

determines, based on the display instruction information 
Cd for all groups (the initialized all groups Gdji, Gdj2, 
...) in the page j, arrangement positions and display 
states of media information and the like to be displayed 

10 on the display screen, and displays a page screen in 

accordance with all the group data Gdji, Gdj2, ... on the 
display section 12 in an arrangement shown in FIG. 7, for 
example. Although a media display window for displaying 
media information such as video images and score images 

15 (drawing) for the groups is blank in FIG. 7, there is 

also the case where media initial display data prepared 
in advance in the display instruction information Cd is 
displayed. It should be noted that a title of a group 
and brief comments provided by the display instruction 

20 information Cd are displayed in the display area (window) 
marked "other display'' in FIG. 7. 
[0049] 

The distribution processing section CPb transfers 
score files (for one page) designated by the score file 

25 designating information Sd of all of the group data Gdji, 
Gdj2, ... to the score reproducing program executing 
section SP. The score reproducing program executing 
section SP generates score display information (image 
data or drawing data for a score static display part) 

30 representing respective group scores of the transferred 
score files (for one page) SFi, and transfers the score 
display information to the page screen display processing 
section CPc. Responsive to this, the page screen display 
processing section CPc carries out processing of 

35 incorporating static score images expanded based on the 
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score display information into respective group score 
display areas in the page screen to thereby display a 
score image SC such as one shown as a first group 
(reproduction part) of the page screen in FIG. 7. It 
5 should be noted that in the page screen shown in FIG. 7 , 
the score image SC is displayed simply using only a staff, 
with a second and subsequent pages also being simplified 
and displayed on a reduced scale. Also, by operating a 
scroll button in the operator device 11, the user can 
10 scroll this page screen to have a desired group displayed. 
[0050] 

Here, if the user refers to the display of the page 
screen and performs an operation to designate a desired 
group (reproduction part) "k" to be reproduced out of all 

15 the initialized groups 1, 2, ... for the present page j, 

the distribution processing section CPb transfers, to the 
MIDI reproducing program executing section MP, a part of 
the management data MNi corresponding to the designated 
reproduction part, that is, group data Gdjk of the 

20 designated page data PDj corresponding to the group k, 

and a MIDI file MFia (where "a" represents an individual 
file designated within the MIDI files MFi) for one 
reproduction part designated by the MIDI file designation 
information Md of the group data Gdjk, and also transfers 

25 the control designation information Cd and the video file 
designation information Vd of the part of the management 
data MNi corresponding to the designated reproduction 
part, that is, the group data Gdjk, to the video 
. reproducing program executing section VP. 

30 [0051] 

FIG. 6 is a flowchart showing the procedure of the 
setting process (browser process) carried out by the 
management program executing section CP mentioned above. 
When the setting process (browser process) is started in 
35 accordance with the management program, first the read 
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processing section CPa reads the designated contents CFi 
from the contents filing section 4F and records the 
contents CFi in the contents buffer 2C (step CI) . Then, 
the data distribution processing section CPb reads all of 
5 the group data Gdjl, Gdj2, ... in the page data PDj of the 
designated page from the management file MNi of the 
content CFi recorded in the contents buffer 2C, and 
causes the page screen display processing section CPc to 
display a page screen in accordance with the group data 
10 Gdjl, Gdj2, ... on the display section 12 (step C2) . 
[0052] 

The data distribution processing section CPb also 
reads the score file designation information Sd from all 
of the group data Gdjl, Gdj2, ... and sends the score file 

15 SFia designated by the score file designation information 
Sd to the score reproducing program executing section SP 
(step C3) . Responsive to this, the score reproducing 
program executing section SP generates score display 
information and sends the score display information to 

20 the management program executing section CP, which 
receives the score display information and displays 
scores in corresponding areas on the page screen based on 
the score display information (step C4) . In this way, 
the scores SC of all of the groups (1, 2,...) included in 

25 the designated page (j) are displayed (see FIG. 7. Note 
that FIG. 7 shows only a simplified display and that at 
this stage, a cursor CL is not shown.) 
[0053] 

Here, if the user performs an operation to designate 
30 or change a group k to be reproduced, the newly 

designated or changed group k is set as an object to be 
reproduced and the following process is carried out for 
the group data Gd j k corresponding to the group k. First, 
processing for detecting file types (MF, AF, VF, SF) to 
35 be used in the reproduction from the group data Gdjl, 
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Gdj2, ... that have been initially designated is carried 
out (step C5) . 
[0054] 

First, it is determined from the designation 
5 contents of the MIDI file designation information Md 

whether a MIDI file (MF) should be used for reproduction 
(step C6) , and when a MIDI file (MF) should be used for 
reproduction ("YES" to the step C6) , the group data Gd j k 
and a MIDI file MFia used for the group k are sent to the 

10 MIDI reproducing program executing section MP (step CI). 
When it is determined that no MIDI file (MF) should be 
used for reproduction ("NO" to the step C6) , the group 
data Gd j k is sent to the MIDI reproducing program 
executing section MP (step C8). 

15 [0055] 

After the data has been transmitted to the MIDI 
reproducing program executing section MP in this way 
(steps CI, C8) , the designation contents of the file 
designation information Vd is checked to determine 

20 whether it has been detected that a video file (VF) 

should be used for reproduction (step C9) . Here, when it 
is determined that a video file (VF) should be used for 
reproduction ("YES" to the step C9) , processing for 
sending the video file designation information Vd in the 

25 group data Gd j k to the video reproducing program 

executing section VP is carried out (step C10), and then 
the setting process (browser process) is terminated. On 
the other hand, when it is determined that no video file 
(VF) should be used for reproduction ("NO" to the step 

30 C9) , the setting process is terminated immediately. 
[0056] 

(2) Processing by Respective Reproducing Program 
Executing Sections MP to SP In Setting Operation Mode 
Next, the operations of the respective media 
35 information reproducing program executing sections MP to 
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SP in the setting operation mode of the performance 
information reproducing apparatus will be described. In 
FIG. 5, first, in the score reproducing program executing 
section SP, the score display processing section SPa 
5 generates the score display information (image data or 
drawing data for the score static part) based on the 
score file SFi (for one page) sent from the data 
distribution processing section CPb of the management 
program executing section CP as described above, and 

10 transfers the score display information to the page 

screen display processing section CPc of the management 
program executing section CP. Further, the score 
reproduction data generating section SPb generates score 
reproduction data by extracting a data part of a movable 

15 display part (note/rest event) of the score, out of the 
score information of one reproduction part of the score 
file SFi (for one page) corresponding to the group k, and 
transfers the generated score reproduction data to the 
reproduction data generating section MPa of the MIDI 

20 reproducing program executing section MP. 
[0057] 

In the MIDI reproducing program executing section MP, 
the reproduction data generating section MPa generates 
audio reproduction data and MIDI reproduction data based 

25 on one reproduction part (group data) Gd j k of the 
management file Mni and the MIDI file MFia of one 
reproduction part transferred from the distribution 
processing section CPb, and also generates reproduction 
data RD as shown in FIG. 8(a), for example, by combining 

30 the above generated data with the score reproduction data 
from the score reproduction data generating section SPb. 
The reproduction data RD generated by the reproduction 
data generating section MPa is stored in the reproduction 
data buffer 2M by the reproduction data storage section 

35 MPb. 
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[0058] 

The reproduction data generating section MPa 
generates audio reproduction data ATr synchronous with 
MIDI clock timing based on the audio file designation 
5 information Ad (see FIG . 3) of the designated group data 
Gd j k of the management file MNi. As is learned from this, 
during a setting operation by the image information 
reproducing apparatus, the audio reproducing program 
executing section AP receives no data distributed from 
10 the distribution processing section CPb, and therefore 
the audio reproducing program executing section AP does 
not carry out any processing on contents data. 
[0059] 

On the other hand, as for the video file designation 
15 information Vd (see FIG. 3), to reduce a burden upon the 
reproduction data generating section MPa of the MIDI 
reproducing program executing section MP, the video file 
designation information Vd (see FIG. 3) is processed by 
the video reproduction data conversion/storage section 
20 VPa of the video reproducing program executing section VP 
in this example. That is, the video reproduction data 
conversion/storage section VPa generates video 
reproduction data VTr as shown in FIG. 8(b) from the 
video file designation information Vd of one reproduction 
25 part (the designated group data Gd j k of the management 
file MNi) distributed from the distribution processing 
section CPb, and stores the generated video reproduction 
data VTr in the video control data buffer 2V. 
[0060] 

30 The reproduction data RD generated by the 

reproduction data generating section MPa and stored in 
the reproduction data buffer 2M by the reproduction data 
storage section MPb is described, for example, in the SMF 
(Standard MIDI File) format in accordance with the 

35 musical tone information format of the MIDI file MF, and 
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in the example in FIG. 8(a), is comprised of reproduction 
master track information CTr, score MIDI track (score 
track) information STr, MIDI track information MTr, and 
audio track information ATr. The video reproduction data 
5 generated by the video reproduction data 

conversion/storage section VPa and stored in the video 
control data buffer 2V is also described in the SMF 
format, as shown in FIG. 8(b). 
[0061] 

10 The score track information STr is score 

reproduction data that is used itself in reproduction of 
a score. The MIDI track information MTr is MIDI 
reproduction data that is used itself in reproduction of 
musical tone information. That is, as shown in FIG. 9(1), 

15 the both types of track information STr, MTr are each 
comprised of pairs of MIDI clock timing MCpl, MCp2 , 
and MIDI events EVpl, EVp2, ... that should be generated in 
that MIDI clock timing, the pairs being arranged in the 
order of the MIDI clock timing. In the present 

20 embodiment, the score track information STr is generated 
by extracting display position information (such as DP11 
in the lower-right part of FIG. 4) of note/rest events 
(movable parts) from score reproduction data, that is, a 
score file SFia and converting the information into the 

25 MIDI clock timing. Such score track information STr is 
used to schematically display progress of performance of 
music tone information of the MIDI track information MTr 
in terms of movement of the cursor CL (see FIG. 7). 
[0062] 

30 On the other hand, the audio track information ATr 

is audio reproduction data that specifies reproduction 
start timing of sound wave information in terms of MIDI 
clock timing and designates audio files AFia, AFib, ... to 
be reproduced in MIDI clock timing. That is, as shown in 

35 FIG. 9(2), the audio track information ATr is comprised 
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of pairs of MIDI clock timing MCql, MCq2, and 
designated audio filenames AFql, AFq2 , ... to be reproduced 
in that MIDI clock timing, the pairs being arranged in 
the order of the MIDI clock timing. 
5 [0063] 

The respective MIDI clock timing MCql, MCq2, ... are 
obtained by converting timing data Tml, Tm2, ... of the 
audio file designation information Ad of the group data 
Gdjk to MIDI clock. The respective designated audio 

10 filenames AFql, AFq2, ... are copied from designated 
filenames Fnl, Fn2, ... of the audio file designation 
information Ad and therefore information that designate 
individual audio files AFia (the contents of which is 
shown in the second part of the right column in FIG. 4) 

15 described above. 
[0064] 

As is the case with the track information ATr for 
audio reproduction data, the video reproduction data in 
FIG. 8(b) is track information VTr in which video files 

20 are associated with MIDI clock timing. As shown in FIG. 
9(3), the video track information VTr is comprised of 
pairs of a MIDI clock timing MCrl, MCr2, and 
designated video filenames VFrl, VFr2, ... that are to be 
reproduced in that MIDI clock timing, with such pairs 

25 being arranged in the order of the MIDI clock timing. In 
the same way as the audio track ATr, the video track 
information VTr is generated from the video file 
designation information Vd of the group data Gdjk. That 
is, the MIDI clock timing MCrl, MCr2, ... are obtained by 

30 converting timing data Tml, Tm2 , ... of the video file 

designation information Vd to MIDI clock timing, and the 
designated video filenames VFrl, AFr2, ... are copied from 
the designated filenames Fnl, Fn2, and therefore 

designate individual video files VFia (the contents of 

35 which is shown in the third part of the right column of 
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FIG . 4) . 
[0065] 

(3) MIDI Generation Processing by MIDI Reproducing 
Program Executing Section MP 
5 FIGS. 10 and 11 are flowcharts showing the procedure 

of a MIDI generation process carried out by the MIDI 
reproducing program executing section MP as described 
above. When the MIDI generation process is started 
according to the MIDI reproducing program, first the 

10 reproduction data generating section MPa of the MIDI 
reproducing program executing section MP reserves the 
reproduction data buffer 2M, which is an area for 
recording the reproduction data RD, in the processing 
buffer (RAM) 2 (step Ml) . Next, processing for detecting 

15 file types (MF, AF, VF, SF) to be used in reproduction 

from the media designation information Md, Ad, Vd, and Sd 
of the designated group data Gd j k from the distribution 
processing section CPb is carried out (step M2) . 
[0066] 

20 First, it is determined whether it has been detected 

that a MIDI file (MF) is to be used for reproduction, 
from the designated contents of the MIDI file designation 
information Md (step M3) . Here, when a MIDI file (MF) is 
to be used for reproduction ("YES" to the step M3) , 

25 information on the master track CT in the MIDI file MFia 
designated by the MIDI file designation information Md is 
set as the master track information CTr of the 
reproduction data RD, and the master track information 
CTr is recorded in a master track of the reproduction 

30 data buffer 2M by the reproduction data storage section 

MPb (step M4) . Further, the master track information CTr 
information on the reproduction track RT in the MIDI file 
MFia is recorded in a MIDI track of the reproduction data 
buffer 2M as the MIDI track information MTr by the 

35 reproduction data storage section MPb (step M5 in FIG. 
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11) . 
[0067] 

When it is determined that no MIDI file (MF) is to 
be used for reproduction ("NO" to the step M3 in FIG. 10), 
5 master track information CTr is newly generated according 
to timing setting information in the control instruction 
information Cd and the generated master track information 
CTr is recorded in the master track of the reproduction 
data buffer 2M by the reproduction data storage section 
10 MPb (step M6) . 
[0068] 

After the master track information CTr has been 
recorded (the steps M5, M6) , it is determined whether it 
has been detected that an audio file (AF) is to be used 

15 for reproduction, from the designation contents of the 
audio file designation information Ad (step M7 in FIG. 
11) . Here, when it is determined that an audio file (AF) 
is to be used for reproduction ("YES" to the step Ml), 
the audio file designation information Ad is read from 

20 the group data Gd j k (step M8) and MIDI clock timing data 

MCql, MCq2, ... are generated from the timing data Tml, Tm2, 
... of the read audio file designation information Ad (step 
M9) . Further, audio track information ATr comprised of 
the MIDI clock timing data MCql, MCq2, ... and designated 

25 filenames AFql, AFq2, ... is generated and recorded in an 
audio track of the reproduction data buffer 2M by the 
reproduction data storage section MPb (step M10) . 
[0069] 

After the audio track information ATr has been 
30 recorded (the step M10) or when it is determined that an 
audio file (AF) is not to be used for reproduction ("NO" 
to the step M3) , it is determined whether it has been 
detected that a score file (SF) is to be used for 
reproduction, from the designation contents of the score 
35 file designation information Sd (step Mil) . Here, when 
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it is determined that a score file (SF) is to be used for 
reproduction ("YES" to the step Mil) , score reproduction 
data RD is received from the score reproduction data RD 
generating section SPa of the score reproducing program 
5 executing section SP and the received score reproduction 
data is recorded in a score track of the reproduction 
data buffer 2M as the score MIDI track information STr by 
the reproduction data storage section MPb (step M12) . 
After the recording processing for the score MIDI track 

10 information STr has been carried out (the step M12) , the 
MIDI generation process is terminated. Also, when it is 
determined that no score file (SF) is to be used for 
reproduction ("NO" to the step Mil), the MIDI generation 
process is terminated immediately. 

15 [0070] 

[Reproduction Operation] 

During a reproduction operation, the performance 
information reproducing apparatus according to the 
embodiment of the present invention uses the reproduction 

20 data RD generated as described above during a setting 
operation to carry out synchronized reproduction of 
various performance information (media information) 
according to a MIDI file MF and other media files AF, VF, 
and SF in a reproduction manner corresponding to the 

25 contents of a management file MN. FIG. 12 is a 

functional block diagram [2] schematically showing 
functions of the performance information reproducing 
apparatus that are used during a reproduction operation 
according to the embodiment of the present invention. 

30 [0071] 

After the operation settings have been completed, 
when the user operates a reproduction switch RS on the 
page screen (see FIG. 7) using a predetermined operator 
of the operator device 11 to give an instruction for. 
35 starting reproduction, a start instruction processing 
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section CPd of the management program executing section 
CP, which gives instructions for starting and stopping a 
reproduction operation of the performance information 
reproducing apparatus, instructs the MIDI reproducing 
5 program executing section MP to start reproduction and 
hence causes the performance information reproducing 
apparatus to start a reproduction operation. 
[0072] 

In the MIDI reproducing program executing section MP, 
10 responsive to this reproduction start instruction, a MIDI 
reproduction start processing section MPc, which gives 
instructions for starting and stopping operation of the 
MIDI reproducing program executing section MP, activates 
a MIDI clock source MC and a MIDI reproduction processing 
15 section MPd. The MIDI reproduction processing section 

MPd reads the reproduction data RD from the reproduction 
data buffer 2M in accordance with a MIDI clock from the 
MIDI clock source MC. 
[0073] 

20 That is, whenever each MIDI clock timing MCpl, MCp2, 

... in the MIDI track information MTr of the reproduction 
data RD is reached, the contents of MIDI event data EVpl, 
EVp2, ... in the same track information MTr is outputted to 
the tone generator circuit 7 and the effect circuit 8 as 

25 musical tone information. At the same time, an operation 
start instruction is issued to a start/data transfer 
section MPe, and whenever each MIDI clock timing MCql, 
MCq2, .... in the audio track information ATr and MIDI clock 
timing MCpl, MCp2, ... in the score track information STr 

30 of the reproduction data RD are reached, a designated 
audio filename AFql , AFq2, ... and a score MIDI (cursor) 
event EVpl,. EVp2, ... are sent to the start /data transfer 
section MPe. 
[0074] 

35 In accordance with control instructions from the 
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MIDI reproduction processing section MPd, the start/data 
transfer section MPe instructs the other type media files 
AP, VP, and SP to start and stop operations. When an 
operation start (activate) instruction has been issued, 
5 the start/data transfer section MPe sends the designated 
audio filenames AFql, AFq2, ... and the score MIDI events 
EVpl, EVp2, ... respectively to the audio reproducing 
program executing section and the score reproducing 
program executing section AP, SP in the respective MIDI 

10 clock timing mentioned above. Also, in accordance with a 
reproduction start instruction from the MIDI reproduction 
start processing section MPc, a MIDI clock is transmitted 
from the MIDI clock. source MC to the video reproducing 
program executing section VP. 

15 [0075] 

An audio reproduction processing section APa of the 
audio reproducing program executing section AP starts 
operation according to the activate instruction from the 
start/data transfer section MPe. Whenever each MIDI 

20 clock timing MCql, MCq2,... is reached, one of audio files 
AFia, AFib, ... (based on the audio file designation 
information Ad of the page data Pd j k in the contents CFi) 
designated by a corresponding one of the designated audio 
filenames AFql, AFq2, ... that have been received is read 

25 from the contents buffer 2C via an audio file reading 
section APb and an audio/video file read processing 
section CPe of the management program executing section 
CP, and the contents of the audio file AFia, AFib,... is 
outputted to the effect circuit 8 as sound wave 

30 information. 
[0076] 

A video reproduction processing section VPc of the 
video reproducing program executing section VP starts 
operation according to the activate instruction from the 
35 start/data transfer section MPe, reads video reproduction 
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data, that is, the video track information VTr, from the 
video control data buffer 2V in accordance with the MIDI 
clock from the MIDI clock source MC, reads a video file 

(VF) corresponding to the read video track information 
5 VTr, and transfers the video file to a video/score 

display processing section CPf of the management program 
executing section CP. 

[0077] 

That is, whenever each MIDI clock timing MCrl, MCr2, 
10 ... in the video track information VTr, STr is reached, the 
video reproduction processing section VPc reads out from 
contents buffer 2C one of video files VFia, VFib, ... 
(based on the video file designation information Vd in 
the page data Pd j k of the contents CFi) designated by a 
15 corresponding one of designated video filenames VFrl, 
VFr2, ... in the video track information VTr, STr via a 
video file reading section VPd and the audio/video file 
read processing section CPe, and outputs the contents of 
the video file VFia, VFib, ... to the video/score display 
20 processing section CPf as image information. 
[0078] 

A score reproduction processing section SPc of the 
score reproducing program executing section SP starts 
operation according to the activate instruction from the 

25 start/data transfer section MPe, and whenever each MIDI 

clock timing MCpl, MCp2, ... in the score track information 
STr is reached, outputs the contents of one of the 
received score MIDI (cursor) event data EVpl, EVp2 , ... to 
the video/score display processing section CPf as movable 

30 score (cursor) information. 
[0079] 

The video/score display processing section CPf of 
the management program executing section CP displays 
images and a movable score (cursor) based on the image 
35 information and the movable score information received 
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from the video reproduction processing section and the 
score reproduction processing section VPc, SPc in 
respective display areas (windows) of the page (j) in the 
page screen. For example, for the first group (k=l) in 
5 the page screen in FIG. 7 , in accordance with the movable 
score information, a cursor image CL representing the 
present musical tone performance position is displayed in 
synchronism with the reproduction operation. It should 
be noted that the display manner of the displayed movable 

10 score is not limited to a cursor image, it is also 

possible to use a method in which a flashing display is 
used for a note/rest image of a note or a rest being 
reproduced in the displayed score, or the color or 
luminance of the present note/rest image is changed for 

15 emphasis . 
[0080] 

Also,- during reproduction and after the reproduction 
has been completed, if the user operates a stop switch SS 
on the page screen using a predetermined operator of the 

20 operator device 11 to instruct stop of a reproduction, 
the start instruction processing section CPd of the' 
management program executing section CP issues an 
instruction to stop the reproduction operation of the 
performance information reproducing apparatus, so that 

25 the MIDI reproduction start processing section MPc of the 
MIDI reproducing program executing section receives the 
instruction and instructs the MIDI clock source MC and 
the MIDI reproducing program executing section MP to stop 
operation, so that the reproduction operation of the 

30 performance information reproducing apparatus is 

terminated. In this case, the reproduction operation can 
be restarted by another operation of the reproduction 
switch RS, or another contents (i), page (j), group (k) 
can be designated and a new setting operation can be 

35 carried out. 
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[0081] 

[Another Embodiment] 

Although in the embodiment described above, the 
video reproduction data (video track information) VTr out 
5 of the reproduction data RD is generated by the video 
reproducing program executing section, the MIDI 
reproducing program executing section MP may generate 
reproduction data RD including the video track 
information VTr. FIG. 13 is a functional block diagram 
10 [3] schematically showing functions of the performance 
information reproducing apparatus that are used in a 
setting operation according to another embodiment that is 
applied in such a case. 
[0082] 

15 In the performance information reproduction system 

according to the other embodiment, the video reproducing 
program executing section VP has the same construction as 
the audio reproducing program executing section AP. In 
the other embodiment, during a setting operation, the 

20 video reproduction data (video track information) VTr is 
generated and stored by the MIDI reproducing program 
executing section MP, so that as shown in FIG. 13, no 
data is distributed to the video reproducing program 
executing section VP from the distribution processing 

25 section CPb and no processing is carried out for contents 
data by the video reproducing program executing section 
VP. Accordingly, there is no need to reserve the video 
control data buffer 2V (see FIG. 12) in the processing 
buffer (RAM). 2 as in the previous embodiment. 

30 [0083] 

On the other hand, the reproduction data generating 
section MPa of the MIDI reproducing program executing 
section MP also generates the video track information VTr 
synchronous with MIDI clock timing based on the video 
35 file designation information Vd in the designated group 
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data Gd j k in the management file MNi, in the same way as 
the audio reproduction data (audio track information) ATr. 
That is, in the reproduction data generating section MPa, 
date reproduction data RD including all of the media 
5 track information MTr , ATr, VTr, and STr corresponding to 
all of the media files MF, AF, VF, SF is generated, as 
shown in an upper-left part of FIG. 13 (in which the same 
contents as in FIG. 9 are shown) . 
[0084] 

10 Then, description will be made with reference to FIG. 

12, during the reproduction operation of the performance 
information reproducing apparatus, the video reproduction 
processing section VPc of the video program executing 
section AP receives an activation instruction from the 

15 start/data transfer section MPe, starts operation, 

receives the video track information VTr, and carries out 
the same operation as the audio reproduction processing 
section APa . Therefore, the video reproducing program 
executing section VP does not need to be supplied with a 

20 MIDI clock from the MIDI clock source MC. (It should be 
noted that there are no other particular differences in 
structure from the example in FIG. 12.) 
[0085] 

That is, whenever each MIDI clock timing MCrl, MCr2, 
25 ... in the video track information VTr is reached, one of 
video file VFia, VFib, ... designated by the corresponding 
designated video filename VFql, VFq2, ... sent from the 
start/data transfer section MPe is read from the contents 
buffer 2C via the video file reading section ( VPd) and 
30 the audio/video file read processing section CPe of the 

management program executing section CP, and the contents 
of the read video files VFia, VFib, ... is outputted to the 
video/score display processing section CPf as image 
information . 
35 [0086] 
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[Various Embodiments ] 

Although the present invention has been described 
according to the certain embodiment, but it can be 
implemented in various ways. For example, although a 
5 single file is designated as each of the musical tone 
information and the score information for one 
reproduction part according to the first embodiment, a 
plurality of files may be designated as each of the 
musical tone information and the score file for one 

10 reproduction part. That is, if the MIDI file designation 
information and the score file designation information Md, 
Sd have the same structure as the audio and video file 
designation information Ad, Vd shown in the lower-right 
part of FIG. 3, it is possible to designate a plurality 

15 of MIDI files MFia, MFib, ... and score files SFia, SFib, ... 
(where "a", "b",... express individual files out of a 
plurality of files designated as the files MFi, SFi) for 
one reproduction part using the group data Gd j k for a 
single group. In this case, the MIDI clock timing (see 

20 FIG. 9) in the MIDI and score track information MTr, STr 
of the reproduction data RD corresponding to the MIDI 
files MFia, MFib, ... and score files SFia, SFib, ... are 
shifted by timing Tml, Tm2, ... of the MIDI file 
designation information and the score file designation 

25 information, Md Sd. 
[0087] 

[Advantages of the Invention] 

As described above, according to the present 
embodiment, without carrying out difficult processing 

30 such as specially generating various media information in 
advance so that the media information is reproduced in 
accordance with reproduction timing of musical tone 
information, when reproduction is carried out, the 
management file is used to easily generate reproduction 

35 data for reading the various media information and the 



40 

various media information can then be read out in 
accordance with the reproduction data only by setting 
read manners (filenames, timing, and the like) for 
various media information in advance in a management file. 
5 As a result, synchronized reproduction of media 

information can be realized with ease. Further, an 
editing operation for a management file that specifies 
reproducing manners of the various media information is 
simple, so that by editing the management file, for 
10 example, to change some of the data files or add new data 
files, or to change the reproduction timing of respective 
media files, it is possible to flexibly set a desired 
reproducing manner. 

[Brief Description of the Drawings] 
15 [FIG 1] FIGS. 1 are block diagrams showing examples of 
the entire construction of a performance information 
reproduction system according to an embodiment of the 
present invention . 

[FIG 2] FIG. 2 is a block diagram showing the hardware 
20 construction of the performance information reproducing 
apparatus according to an embodiment of the present 
invention . 

[FIG 3] FIG. 3 is a diagram illustrating the exemplary 
structures of contents files used in performance 
25 information reproduction and management data according to 
an embodiment of the present invention. 

[FIG 4] FIG. 4 is a diagram illustrating the exemplary 
structures of respective media files (for one file) used 
in performance information reproduction according to an 

30 embodiment of the present invention. 

[FIG 5] FIG. 5 is a functional block diagram [1] 
schematically showing functions of the performance 
information reproducing apparatus in a setting operation 
mode according to an embodiment of the present invention. 

35 [FIG 6] FIG. 6 is a flowchart showing a setting process 
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(browser process) carried out during a setting operation 
of the performance information reproducing apparatus 
according to an embodiment of the present invention. 

[FIG 7] FIG. 7 is a view showing an example of a page 
5 screen displayed during a setting operation of the 

performance information reproducing apparatus according 
to an embodiment of the present invention. 

[FIG 8] FIGS. 8 are diagrams showing an example of the 
overview of reproduction data RD generated during the 
10 setting operation of the performance information 

reproducing apparatus according to an embodiment of the 
present invention. 

[FIG 9] FIGS. 9 are diagrams showing an example of the 
contents of the reproduction data RD generated during the 
15 setting operation of the performance information 

reproducing apparatus according to an embodiment of the 
present invention . 

[FIG 10] FIG. 10 is a part of a flowchart showing a MIDI 
generation process carried out during the setting 
20 operation of the performance information reproducing 
apparatus according to an embodiment of the present 
invention . 

[FIG 11] FIG. 11 is another part of the flowchart showing 
the MIDI generation process carried out during the 
25 setting operation of the performance information 

reproducing apparatus according to an embodiment of the 
present invention . 

[FIG 12] FIG. 12 is a functional block diagram [2] 
schematically showing functions of the performance 
30 information reproducing apparatus that are used in a 

reproduction operation mode according to an embodiment of 
the present invention. 

[FIG 13] FIG. 13 is a functional block diagram [3] 
schematically showing functions of a performance 
35 information reproducing apparatus according to a second 
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embodiment of the present invention that are used in a 

setting operation mode according to an embodiment of the 

present invention . 

[Description of Symbols] 
5 CF: CFl-CFn; CFi contents file 

MN: MN1; MNi management data 

PD1; PD2; PDj page data 

Gd: Gdll, Gdl2; Gd21, Gd22; Gd j k 

MF: MF11, MF12; MFi; MFil, MFi2 
10 tone information file) 

' AF: AF11, AF12; AFi; AFil, AFi2 

wave information file) 

VF: VF11, VF12; VFi; VFil, VFi2 

information file) 
15 SF: SF11, SF12; SFi; SFil, SFi2 

CP management program executing section 

MP, AP, VP, SP MIDI reproducing program executing 

section, audio reproducing program executing section, 

video reproducing program executing section, score 
20 reproducing program executing section 

RD reproduction data 



group data 
MIDI file (musical 

audio file (sound 

video file ( image 

score file 
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[Name of the Document] Abstract 
[Abstract] 

[Problem to be Solved] To achieve easy generation of 
reproduction data and synchronized reproduction of a 
5 plurality of types of performance information. 

[Solution] In a performance information reproduction 
system according to the present invention, various types 
of media information (performance information) data files 
MF, AF, VF, and SF are stored in the contents filing 

10 section 4F or the like together with management files MN 
that manage the reproduction manners of the data files 
MF-SF. Whenever synchronized reproduction of such media 
information is carried out, a reproduction data 
generating part MPa, etc. of a musical tone information 

15 reproducing program executing section MP generates, in 

accordance with the management file MN, reproduction data 
RD in the state for enabling musical tone data files and 
score data files MF, SF, and various other media data 
files AF, VF to be read out according to a musical tone 

20 information reproducing program. Then, reproducing parts 
of respective media information reproducing program 
executing sections MP, SP; AP, VP reproduce the musical 
tones and score information of the reproduction data RD 
and also the other media information of the data files AF, 

25 VF read out based on the reproduction data RD. 
[Selected Drawing] Figure 5 



